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(54) DIGITAL COMPOSITION OF MOSAIC IMAGE 

(57) Abstract: 

PROBLEM TO BE SOLVED: To generate a mosaic 
image approximating a target image from a data 
base for a source image by generating mosaic 
images having source images which are arranged 
at the respective tile parts of mosaic images 
corresponding to respective tile parts and best 
match them. 

SOLUTION: A computer workstation 20 acquires a 
source image 12 from a video tape player 16 and a 
laser disk player 18. Further, the computer 
workstation 20 inputs a target image 14 and 
generates a mosaic image 10 from the target image 
14 and the source image 12. The mosaic image 10 
generated by mosaic software includes an array of 
tiles 22. Each tile 22 is a source image 12 and the 
whole outward appearance of the mosaic image 10 
approximates the outward appearance of the target 
image 14. 
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(54) [«95£>£»] ^ef-V^ • y — ^(D'ri^^/v • 3>-3Kv?^3>- 



(57) [Kfi] 

[MS] y-y • t—w*?— ^-<— x^e><o^ 

. ^^^^ 

hcom&&<D^^y-* • ^y — ^m^s 




( 2 ) 

1 

[if*^l] 1g«W>y— * ■ -f ^Srfflv^5^ biz 
$s l fciE®8E#-e fo o T , 

^-yy^b^m^.. 10 

^-Y/wffittiJtUE-rsJtiSE^yy^-cfcor, #y-y 

y-y • -Y^-^Sr^W-rSiM^Xxs/^i:, 
ig^^ttfcSlliEy— * -Wy — MfE^eif W ^ • W 

\^m-t^^y-y^t^m^^m^rfo i. okmmz* 20 

[ff*^2l 111**1 IE«wflB«IIE#fci3V^-C. MSB 

5:fi*wy-x • Wy— ^#<fcJ£t^r£y^s/:7°i:£: 

Q-y-ym^fry loo^^t/^ttsv^ • wy 

looy-^ • y — ^^e-tf-f ^ • -ry— -^iwio^T 
1 la] J: 9 #5- < Sift 3 r tofti,^ 5tw, gfJlBittiRyxs/ 40 
^fcJS^-CjB^Sftfcy— * • Wy-v'^It/^bKK 

MfEffiffiyx^^^Tffiffi^tifc y—y -wy — 50 
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2 

«:*JKfc«I9Bt5rt^J:0, SEJESitfc y— ^ • W y 
[ft** 8 1 W*3S7Ett©1E«afltfc*5V^-C, mJSH 

«*E$ttfc-f y-^Sr^'L^b^JSa^^yySrUw^- 
[if*iS9] »**8SEtt«?8E«ai*:Jw*5V^T v fltrE 

B«MS*tw*itie£;h,3ty— • y-^Ji^t^ t£ 
JitE £ *t7cW y - v=Sr x k> ±frt>$) *) z y 7- v zr 

[1*«10] »**7|B<6©fEftja*^Jsv^-C. ml 

-r— ^-<— ^rt^WtE^ttfcy— y •-I'y—^^ 

[ft**12] »#*llE#«Efitfi#:»::*5^-C\ ffi 

755, s*«j»isn4w**©jii[t«foi«riEy-^ -wy-- 
UtftlEy— • -f y— *Jifim<n9 ^ j\>mmz.%i 

7t*tc^ft< itiooy— y. ♦ -fy — ^S-Jg^-f-^y. 

1 4 J If 1 |B^ro|E^^(C*3V vt\ ffj 

[o o o 1 1 

y — v ; w^iw^-r.5 <> 

[00 0 21 

JBya^wwy-^^^w-r-awi-fflv^ft-tr^TCo # 



( 3 ) 

3 

[0 0 0 3] 

y— ;* ^^-yt^-^y h * -fy-wwsixfe 

^ »*3it/# (RGB) oa*5jl*9»M(05p^tt 
(Root-Hean-Square (RMS) error) 3 ^Srff^ 

*«*ttfc*t>Sffiii-5 y-* • >r y -^sraoitm-r 

[0 0 0 4] --fy — v^c 

• >f y - 5* fc*5 5 *■ * >f ;vW£ y « W — $ 
*X. RGBORMSX7- (RMS error) # 

vtim^x, y~ymm*&y-* -^y-^ttje-r* 

$ tt/t RGB<7)RMS^7 ^-^AD^ $ ttT x y—* • >f 
y — i?±mzMTZ>&iYR G B RMS xy^-^gftf 
So */K<7>&frRGB Sr#o« 5 gt?) 30 

nrv^wy — ^f-V^ • ^ ^-^tcjott^^- 

£fflV^ 0 40 
[0 0 0 5] 

4K3ttffil-rSJ:5^ fflffi^Hfcy-^ -^y-^i 2 

— * . >(y — ^iiJESrSS^i-Sfc*. VHSfft 
. ^-^^ y— * • >r *—i?b b"Cfflt^bix^»it 
^^^y^n^/c^tcffl^btt^o y— ^ 50 
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4 

— ff^f • iri 8 <>*fc, •/f>-> f 
JH^t^^^coy— ^^bA¥^IB"t?*>S^ttu^ 

by-;* ->f y— ^1 2§rtt8H-5fc*/ tr-r^A^S: 

6 Q ^yba^ . y—v^^—'y^ 1/2 o A 
h ->f y — v?l 4 

y—i/t y— ;* • ^ y — v^^b^fM* ^ • y — ^1 
TtdiD^sixs^if-Y^ • y— oji^>r^2 
y — v^i 2T?fo»9, ^ef-v^ * >r y — >>i 0(t?^-m^ 

y^^-hi/g^/ (Adobe Photoshop) (3£ifci&gg) 

v"y^ (9ftffi«0 , PC^fciiUNlx («£Sfg9I) 
icS< iy^^Atoj: 5*11* ^ vt°^-^ 2 4 £\ ^eif 

• *f y — 2^lim-r^fc^{cfflv^^r 

[0006] JtlEStifcy— ^ • y — v?i 2tt. 5>W 

^3o^ia«Six5rfc36SBrtB-efc5. ^<7?ffiffiLfcy 

• wy— v^i 2S:^«ftt^ft^b»ffc4y-^ • 

y ^^r^fej^ci^^) fc^>[^. add images t 

o database '("f y — v^Sry 5 "^ ^ ^ 

AO) ^n^i^JBi^^^ <t9f¥*IBi-tt. add_ 
ima ge s t o d a t a b a s e^u^^AI^ 

r^AE^ix/c^x^ hysri^ty^ -^y-v* 
Srf— ^-f-s J; 5»f^u, r<^>y— ^ • >f y— v?^bJt 

» (square) *Sa*»K«w i 9 «£SftS»9f— 

fctf, jE^-^y — ^sy— ^ * y — ^(O^^^b^J 
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5 

■efc%5 J:5(-*So y—i^-r — 3 

Otcl&iWSiX'So -r— 3 0M\ £ Xtr 

[0007] 0 2it f^^3 0 (Hi) fa<oy 

— ^ • 4*— isl 2<D^i^Sr^i- 0 y— *.>fy — v?i 
2 12, Sj^A— h * y — K 3 2 , h • y — K 3 

4 , mm^— K • / — K 3 6 OiP t /U— h • / — K (ro 
ot node) <DTK#S£ftTEH£;h,5o tM^coy*-^ 10 

i*. Kj^z/u- h . y~ K3 2^s^-tf>f ^ • yy h 

^3*U-C«KSixS 0 h • y- K3 2CQT^fe 

• y r^fc^try-^-r-r h y -c*>5 0 
^y — oco^ut) ^^tt-cv^/^t^— vsr^t^o 

add image s t o d a taba s ^~f^{f 

y v^/U • ^zff^i l/^M3 8 ^*6«FStt5o 2 5 6 
X 2 5 6 (t°^-tr/u) 4 2*5^^6 4 X6 4 (t°^ir 

zu) 4 4-e*$ti^-7 2 V h y 12. 7^-7^HkS 

ffll^Mc *fi?!lic*5V^-Ctt, 3 2 X 3 2 (tT^-fe/u) 
4 6 (Dy"j bV il. i|/nt^«i;^^ y —V 

y — ^Sr^tPo 16X16 (tT^-fe/u) 48. 8X 
struct an^image { 

. chaa: *path; 

char \ised ; 

A» / 

unsigned short *r ; 

unsigned short *g; 
unsigned short *b; 
struct an image *next ; 

> * / 

struct an_image *previous ; / 
} an_image ; 

[ooio] mz-1i* ^eif-T^ • 4 y — */\c&ftZ>%t$ 
>f /u^i 8 Xttt^fflUx 8 Y«t^ffi8S:ttrft 5> tf , 
8X8 (tf^-ir/i') /f^-v/^r-^^^bo-K 

y— v'COINf X|* % gg#-&;bH*: (matching) y'nir^^ 



#^2 000-1 72840 
6 

8 (tr^-fc/U) 5 0, ^cfcU^lXl (tT^-feyu) 5 20 

-y-y-r -f 1^ h y t±, ^ • y y h *^7as$ns!R 

^^tl^^/Un- K (preload) $jx5 y~ * • 
y— v^&r^&v 16X16 (f^-feylO ,8X8 (t°y 
-fe/u) *5J;iM x 1 (t*^i?/u) m^tr-??* V V \z 

-£5 (^#-&fcit-5. matching) fc«>lr/B V n £>;ft,£ 0 fife 

[0 0 0 8] 0 3lt ^e-y-V^ -><^^S:^t5fc 
fc^^rffiSr^-f-o El 2. El3*5j;^0 4Sr#fig"t"5o ^ 

£5t-, ^^<— y,|c*5tt^) y— ^ • y — ^coyt— 
•^^^^ y-<— ^ • y<^\zXV^ 

-So 

[0 00 9] 
/ -f ^ - v?^!ffiffl^*Lfc3ft^^-7 



[0011] ^e-y-V^ * /n^yAfi, y— ^ 
^t^^ (#^) ^n^^^^tr^r^o 
r x j x r yJ ^-f yu2 2 tr^fiJ^tL^o ::t\ 

(x, y) il, tAT<^J: 9"C*)^o ( t a r g e t_ i 

mage w idth / subsamples, 

target image height / hei 



( 5 ) 

7 

gh t sub s amp 1 e s) ((^ — *y y ~- 

^^/U2 2C0^±CDf-y^6 6"e|§g*&U # 

[0 0 12] ^^/U2 2*'>y 

»*3,fctfW (RGB) ^^^^/UCOg^^PM^^ 

«tf>S»is, iS«i*«p«fe-efc 9 rft^ (used) j 

^/U^^t-r^^CO^fh (running sum) iLTffi^^ 

7 0-M^ o &X<D^7ffi$ t &ftm%*lfcb*'rv'7 7 ' 

*"ry-f7 si^sns <t 5^, y— * y 
— ^ft^ffiEi: y— ^ • y— >?fr*fi-s>f 30 

l-o^T##r£n*:B3\ */J^fH([SnfcRGB RM 
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8 

l^l^T, x — ^-<— *K*5tt5*iLtf>y^- * . y — ^ 
5/^8 4fcl*3^T\ fffcfty-;* ->f y— ^n-KS 

fi-^^^-sr^oy-^ • y— sr^/hcw 
y— * ->r y— y— ;* ->r y—v^s^-uv^ 

y— y{e:ilHlJ:9*<3Six*v^J:5^. rft^j 
[0014] h • <4 *~*J\Z3olfZ>£#'{/u\z 

J;5U\ #^>r^i-^LTO*/h09^ft(7?^v--C>y — 

v ^ yi^s * 7c fe-T 5 w £ ri*¥iJ5£ $ ft e> tf > 

[0 0 15] RMSxy- (Ofi-g^^tfS^t^^^r^o 
^y^cDloco/<y^— Sx'g^tt, TIBco J: 5 t-H^^ix 

3 0(01 (^2. *4) iC5>#Jbfc 0 ) 

[0016] 
[*2] 



( 6 ) ifcffl 2000-1 72840 

9 10 

* - ?<om */ 

int f ind_matches (int x, iat y) 

register i, rt, gt, bt; 
int low, result, ii, the tile; 
char imagenarae [256] , best_path [2S6] ; 
EXMAX^xMAX] 1 ^ Sh ° rt rm * S [XMAX*YMAX] , gtnas [XMAX*YMAX} , btnas 

the_tile « x+ (y*sizejc) ; 

/* £ 4x * 0 ^ */ * X 

for(i - 0; i < pixels; { 

tiles (the_tilej .list [ij . scoi:e=99999999 ■ 
tiles [the_tile] .list [i J. rank =* O,- 

strcpy {imagename, filename) ; -/* & — f ? h * J — is <0 

imagenamefstrlen(iinagename) -3J = 's'- /* ^n^jffitO^:^ r-f^x 

imagename [strlen(imagename) - 2j = *g ; ; 

itragename [strlen (imageiicune) -11 = ' 1* : 

getjrid^spacedrmaa, gmaa, bmaa, x, y) ; /* f^Jf^ * — CO 

i" 1 ! 9 ^^ head^image,- */ y _ x H j _ ^ Q ,j v ^ ft ^ ^ ¥j ^ 

^i 1 ®^" 1 ^®-^^ !« NULL) { /* »V-X-f ^-^IC^U-C^^ 
result - 0 ; 

^ tt ^£?£T L '*« ***** »#tt*^7--lc-^vc4D*M-rso 



[0 0 17] 



40 [^3] 
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11 12 
for(i « 0? i > size; i++) { 

rt = (int) ( {unsigned char) rmas(i] - (unsigned 
char) image- >r[i] ) ; 

gt = (int) { (unsigned char)gmas[i] - (unsigned char) 
image- >g[i] ; 

bt = (int) ( (unsigned char)bmasli] - (unsigned. 

char) image - >b [i] ; 

result (rt*rt+gt*gt+bt*bt) ; 

) 

i = 0; 

/* |**> y> (D^iciatfx >y b */ 

if (result < tiles I the__t ale] - list (piacels-1] . score) { 

while ( (result > tiles {the_tile} .list fij - score) 
&&(i++ < pixels) ) ; 

for(ii « pixels -1; ii> i; ii — ) { 

tilegtthe tile] - list [ii) ,scor^ » tilesfthe 
tile] .list [xi-13 . score; 

tiles [the tile] . list [ii] .rank .*= tiles [the 
tile] . list Ui-li - rank; 

tiles (the_t±le) -list [ii3 -pointer = tiles {the 
tile] . list] ii-1] -pointer; 

tiles (the^tile] .list [LJ .score = result; 

tiles [the_tile] * list [i] .rank — i; 

tiles [the_tilel .liot [i] -pointer = image; 

xm *> */ 

image = i mage- > next ; 

} /* wh i 1 e (**7-f /U) */ 

low tiles [the_tile] .list [0] .score; 

tiles [the_tile] .score = tiles [the_tile] . list [0] . score ; 
tiles [the„tile] -rank = tiles (the„tile] . list [0] . rank; 

strcpy(best_path l tiles ft he_tile] -list[0] .pointer- > path) ; 

tiles [the_t lie] .required « tiles (the tilej . list [0] . pointer- 
required; ~~ 



[0018] [^4] 

strcpy{tiles [the_tile] .path, best jpath] : 

sprint f (iraagename, m %s/Zs« . disp_version r bestjpath) ; 

return low; 

} /* f ind_matches () */ 
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13 14 

asi^-c— a (»-&) (Dm^^xvm^htf. tz 0 ) 

tbfcl^w b%m^Z>o (^(DZfusf^J^te^ it^^ls [0 0 2 0] 

^ . /* ^D^7A (find matches ()) Ogl 1 <D7 ^ -X*C:|3^T\ * - 

ft i wot* Fx**-***. ^tZ>m-~§«<Dv-;W;*-^ 

>-5>fcU l<t*>^< coy »J ? FttSlefctLTSffico— &<DgC£^ 

int optimize () 

int. i f x, deepest = o, change, a, step, which? 

for (a = 0; a < pixels; a++) { 
change = 0 ; 

for(x = 0; x < pixels; x++) { 
which =0; 
do { 

step - 0 ; 

for(i = 0; i < pixels; i++) { 

_ /* *?-e**W&o V— 

16 L„ *"/ 

f - ^^-^5:i^t|5o rste p (Xt ^ :/)J *i 0 (cii'^ 

!K */ 

/* do-wh i I e ( F — 

. . if ( (tiles [i] . rank < = 

which) t& ( ! strcmpUiles [x] .list [which] .pointer->patch # 
tiles [i] .path) ) ) { ^ 



[00 2 1 1 



[*6] 
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15 



16 



which) 5t& (tiles [i] .score > 
cont inue 



/* 5>^(raok)^gIi;-?r*tt 

if ( (tiles [x] .rank 
tiles [x] .list [which] .score) } 

if (i == x) continue; 
which++ ; 
step - 1, 

i - pixels; /* wh i 1 e 



} 



} while (step) ; 
if (which > deepest) deepest 



which; 



^MXa\^mu<o j — */ 

£ S3* L > — -J cd^^u * / 
/* *!5oV 

if (strcrop (tiles [x] .path, tiles fx J . list [which] .point er- 
change-*- +; 

strcpy (tiles [x] .path, tiles [x] . list [which] 1 pointer - 
>path) ? 

>requiredT tX} ' T ^ ±1C&d m "les (x) . list [which] -pointer^ 

tiles [x] .score = tiles lx] .list [which] . score - 
tiles fx] .rank = which; 

} /* f o r <v*~) */ 

*y <fc L l 
eOo */ 

if (J change) break; 

fprintf (stderr, «\n%d/%d, %d changes (deepest ia %d)W a 
pixels-l, change, deepest); ; >D ' a ' 

} /* optimize () */ 



[0 0 2 2] ^#^^it•■7 P a^^^cggV^-C^ei^V ^ • 
[0 0 2 3] -rv^^/u. ^eif>f ^ ^^-yij:, n n D fC, 



ft6Sr«<« *Pfl8H^y (solid color) 

COM (Ko-zyO O^SrBV^T, 

*SMPH<7?Ky h§:J8v^;ii:K:«fc ?> fe^WBffSr^-t- 
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18 



jj&xibZo 

[0024] y-^ ^^-yaRwt^tSfc 
-r5#w<z>sa«as. Ei5t3:*£ft-cv^ 0 ^kehtU-v 

^ • >f v^l 0 2. ^2<0^el>M^ *-f ^ — *sl 0 4 
*3J:rJ«^3C9^eif-f ^ • >f v*i 0 6 £^ft^ft£j$ 

^10 St^ft^-f/K *5JzIMt<— v 5 ! 1 0*3«t 

spa&Sr^fc-t'o — i^l 0 8 feitfl 0 2S:tD5fe 
*ift5»l<Z>»#f«-i3V^"C, i^««tt/Bi^e>ftft 
l\> 4 1 0*5<fcWl 0 4 Sr^C 5^*^*5* 

2co^#rf-*^T. ^yvSfc9 4X40f^«^ffl 

rft^<D^£ft/c3i 
«as3fii:£ft5o 20 

[0025] y— v?l 1 2 jo J: 15 1 0 6S:^C5jB* 

V^n^o 16X1 6tys^m W^-^11 
2ht^-^> h • -fy-^i o otrUfCtotriSi^ S 

mm *%«sn5«Ftii, H^s^fts^ 

tf^ttii, V • *< y — v^tOl^J C|g«^*5(t 5 

«E*|6j<^Hv^— tPIC^-Cfc^o Sic. ^iJco^-f/P 

ft^ef-V ^ • -Y y - v^-f y —vm&Mtilztt LT^A 

[0 0 2 6] St^^y-^ •-Y^-^IK(7)l|t 
tcjD^T, -9-^ffl*SS:ffifflb. iSJajft^-f t 1 — ^ (high 

-frequency detail) <7?4>ft I MgtjEtr*!" ^ ^ h 

«K:$WlT£ft5o w^w^li-r^— v*l 0 6^>T^^>8 40 
o<^^/u-ei&a63^ btfX^^ -^ft£>i3:-fy — v*l 0 

[00 2 7] B6lt i? - 4 * — i?M&&\Z&l1- 

5^»coi>-yS«|(ft^)a*S:*UTV^o f lOTf^f 
^ • y — v^l 1 4 £^2<D^eif-f 9 • -f y 1 6 
I*. F^^^ll 8^£/&£ft7c 0 »1 

oo^ef-V ? • *t * —i? l l 4 y— ^ * ^-^M 



t^±9^Sixfco ^2^-e-^V^ "Yy-^l l 6 
^^/urt^l 6x i 6t^««l:«t9 4*Shfeo &i 

y-y> ^(oiac**9^fflv^e>nfc. 
x-ttfc^zhzfj j\s*mTit$>tzg l /£z>y--x -^y 

— ftfco jg^ »2<D^eif>f * ^y-v?i 

l 6tt, 1 co^V ^ ^^-^n 4^9 t>^-^ 
5, h . y-v^l l 8 to3SffiH4^Ht-5fiv\ $£oT, 

[0 0 2 8] ft»WftHJ6^»l»fcj3^"C, *lF>f* ■ 

^#^ttS* (semantic) ^S^ffi^S 

[0 0 2 9] sij^f^#^ft^^^s-efL ^tf-v ^ • 

[0 0 3 0] LV^lSTgffi^ot^T^fc 
ttW6>^-e*^5o tot, **WHt. 5a^$tifc^JS 

co «sh ^ x o r co^-ps^ $ ft 5 1 ^ i aft $ tt 5 -e 
[0ffioffi¥ftt^pj] 

[02] V-X • ^-^^X^y n y ^0 

[0 3] -eif-T^ ^^-^*ffi^t7P-iT' 

[04] ^^>ui:^««i:S:^-rH"CfcS. 

[05] y— ^ • >f V><om$l\^\<^x<n^zfms$.ft 

[0 6] *^ft^eif-r^ ->f ^ — ^co^gtrot^r 
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